It was sometime after the fire of 1871 that Palmer had built his mansion. The location of which was fairly isolated from the downtown business district so it wasn't ideal to live there for many. Nonetheless a few wealthy families did live their at this time, including the Palmer's and the McCormick's.